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(54) Title: TEAT LOCATION FOR MILKING 
(57) Abstract 

A method of guiding a milking appara- 
tus support towards at least one teat of a milk 
animal, characterized in that the method com- 
prises the following steps: to move said sup- 
port to a fixed start position within the stall, 
without any reference to any animal, to illumi- 
nate with a sheet of light from the support a 
region expected to contain ai least one udder, 
to capnire from said support images by viewing 
the region with a video camera, to analyze said 
captured images to identify possible teat candi- 
dates, to select one of said teat candidates as 
a target teat, to determine the position of said 
target teat, and to home in said support and any 
supported milking apparatus to said target teat. 
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TEAT LOCATION FOR MILKING 

TECHNICAL FIELD OF THE INVENTION 

The present invention relates to a method of 
5 guiding a milking apparatus support towards a teat of a 
milk animal and a milking apparatus support guide 
arrangement . 

DESCRIPTION OF RELATED ART 

10 Over some twenty to thirty years, numerous pro- 

posals have been made for techniques by which a milking 
apparatus, defined as including a teat cup connected to a 
milking machine, a teat cleaning device and a teat exami- 
nation device, can be applied to a teat of a milk animal, 

15 such as a cow, by automatic means so as to avoid the need 
for attendance by an operator during the procedure of 
milking an animal. 

As automatic techniques for the rest of the milk- 
ing procedure have been available for some time, automation 

20 of the teat cup application stage has become the main ob- 
stacle to the development of a fully-automatic milking pro- 
cedure which does not require continuous attendance and 
enables the so-called "milking on demand" regime. 

For many reasons, e.g. animal safety and comfort, 

25 milk hygiene and economic efficiency, the application stage 
has to be extremely reliable. That is a teat cup must be 
quickly and correctly applied to a teat on every occasion 
when an animal presents for milking. Also, the equipment to 
carry out the application stage has to work in difficult 

3 0 conditions and must be durable while not being too expen- 
sive. 

EP-A-306 579 discloses a combination of a general 
sensor and local sensor to guide a teat cup onto a teat. 
The general sensor is mounted alongside the stall and pro- 
3 5 jects a sheet of light derived from a laser towards the 

region where the udder and teats of an animal in the stall 
for milking will be present. The sheet of light is incident 

CONFIRMATION 
COPY 
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on an udder and teats when present and produces lines of 
light on the udder and teats. The sheet of light can be 
swept through the region to an appropriate position. A 
camera is mounted as part of the sensor and forms an image 
5 of the lines. The image includes discontinuous lines for 
the teats where illuminated by the sheet of light. These 
lines are analysed in a control arrangement so as to find 
the position of the teat(s) and calculate coordinates by 
triangulation for two horizontal axes and a vertical axis. 

10 Reference values for the coordinates stored in the control 
arrangement assist in analysis by limiting the search and 
calculation time. The calculated coordinates are used to 
bring the local sensors in range of a teat. Knowledge of 
the reference coordinates may be used to limit the image 

15 analysis to the part of the image where the teats are ex- 
pected to be found, thereby reducing the calculation time. 

This disclosure has the disadvantage that two sen- 
sors are needed and that one has to be mounted alongside 
the stall in an exposed position. 

20 

SUMMARY OF THE INVENTION 

It is an object of the invention to provide an im- 
proved teat location technique for automatic milking proce- 
dures, which can provide the complete teat location pro- 
25 cess. 

According to the invention, there is provided a 
method of guiding a milking apparatus support towards at 
least one teat of a milk animal, characterized in that the 
method comprises the following steps: 
3 0 - to move said support to a fixed start position within a 
stall, without any reference to any animal, 
to illuminate with a sheet of light from the support a 
region expected to contain one udder, 
to capture from said support images by viewing the 
35 region with a video camera, 

to analyse said captured images to identify possible 
teat candidates, 
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to select one of said teat candidates as a target teat, 
to determine the position of said target teat, and 
to home in said support and any supported milking 
apparatus to said target teat. 
5 The main advantage with the method according to the present 
invention is that it does not require the use of so called 
historical data regarding the teat positions of the animal 
individualas . The animals do not need to be placed in 
special stalls with tight constraints. They are free to 
10 move within the limits of the stall. 

Advantageously, the method, subsequently to se- 
lecting a target teat, comprises the step: 

to analyse only a subimage of said captured image, 
which subimage includes said target teat. Hereby, the 
15 speeding up of the method is achieved. 

Preferably, the method also comprises the 
following steps: 

to provide in said captured image the position of a 
teat cup entry point for a target teat, 
20 - to quantify the separation of the teat cup entry point 
from the position of said target teat, and 
to provide homing in information for said support on 
the basis of the amount of said separation. 

Advantageously, the method also comprises the 

25 step: 

to quantify said separation in steps defined by areas 
of said captured image. 

Preferably, the method also comprises the step: 
to guide the support to move the subimage of the 
3 0 captures image within the captured image to where the 

entry point for a supported teat cup is at the image 
position of the teat cup entry point for the selected 
teat cup. 

Advantageously, the method also comprises the 

35 step: 

to identify possible teat candidates as respective 
pixels which are adjacent and having a form and position 
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appropriate to a teat illuminated with the light. 

Preferably, the method also comprises the step: 
to identify possible teat candidates by scanning the 
image in a succession of vertical scans to identify 
5 the brightest pixel in each scan 

Advantageously, the method also comprises the 

step : 

to identify the brightest pixel from among a vertical 
line of pixels of differing brightness. 

10 Preferably, the method also comprises the step: 

to identify a possible teat candidate from said 
brightest pixels in adjacent scans which form a 
substantially horizontal line of length in a 
preselected range of pixels. 

15 Advantageously, the method also comprises the 

steps : 

to actuate a region containing said possible teat 
candidates with the aid of an actuating device, so as 
to bring at least one teat to movement, and 
20 - to identify said at least one teat by establishing 
said movement. 

The main advantage with this step is that the 
actuating step makes the teats moving, which makes it 
easier for the system to identify the teats. Another 
25 advantage with this step is that straws can be swept away 
resulting in the elimination of a source of error. 

Preferably, the actuating step in a first 
embodiment, is performed by having said actuating device 
making physical contact with said teat candidates. 
3 0 Advantageously, the actuating step in a second 

embodiment, is performed by having a fluid sprayed towards 
said teat candidates. 

Preferably, the actuating device in a third 
embodiment, is performed by having both a fluid sprayed 
3 5 towards said teat candidates and said actuating device 
making physical contact with said teat candidates. 
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Advantageously, the identifying step is performed 
by identifying said at least one teat by establishing 
character-istic movement thereof. 

This step makes it possible to distinguish a teat 
5 from a leg of a milk animal ^ which leg has another 
characteristic movement. 

Preferably, the method also comprises the step: 
to repeat the above mentioned steps, starting with the 
illuminating step, until each of all the teats of a 
10 milk animal have been selected as a target teat, and 

said support and any supported milking apparatus have 
been homed in to each of all the teats. 

According to the invention, there is furthermore 
provided a milking apparatus support guide arrangement, 
15 characterized in that the support carries a source of a 

sheet of light positioned to pass no lower than a mouth of 
a teat cup arranged on said support and a video camera 
arranged to view through' the sheet and over the mouth of 
the teat cup, wherein said source and video camera are 

2 0 arranged together to cooperate forwardly of the support, 

the video camera arranged to capture an image formed by 
said light forwardly of the support and provide an image 
signal, the arrangement further including image signal 
processing means to analyse said captured image signal and 
25 identify possible teat candidates as a target teat and to 
select one of said teat candidates as a target teat and to 
determine the position of said target teat to home in said 
support and any supported milking apparatus to said target 
teat . The main advantage of the arrangement according to 

3 0 the present invention is that it does not require the use 

of what is called historical data about the animals. The 
animals do not need to be placed in special stalls with 
tight constraints. They are free to move within the limits 
of the stall. 

35 Advantageously, the image signal processing means 

includes means to select from the captured image signal a 
subimage including said target teat and analyse only said 
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subimage, thereby reducing the time between successive 
processings of said subiniage. Hereby, the speeding up of 
the arrangement is achieved. 

Preferably, the image signal processing means 
5 provides, in said captured image, the position of a teat 
cup entry point for a target teat. 

Advantageously, the image signal processing means 
includes means to quantify the spatial separation of the 
teat cup entry point from the position of said target teat 
10 and provide homing in information for said support on the 
basis of the amount of said separation. 

Preferably, the homing in information causes more 
rapid movement of the support, the greater said spatial 
separation . 

15 Advantageously, the arrangement also comprises an 

actuating device for actuating a region containing said 
possible teat candidates, so as to make at least one teat 
moving and a control means to identify said at least one 
teat by establishing the movement. 

2 0 The main advantage is that it is easier for the 

system to identify the teats when they are moving, or after 
they have performed a movement. Another advantage is that 
straws can be swept away resulting in the elimination of a 
source of error. 

2 5 Preferably, the actuating device is .arranged on 

said support. 

Advantageously, in another embodiment, the 
actuating device is arranged on a separate movable support. 

Preferably, in another embodiment, the actuating 

30 device is arranged on the floor in a stall. 

Advantageously, in another embodiment, the 
actuating device is arranged on a post of a stall. 

Preferably, the actuating device is provided with 
spray-nozzles for spraying a flu: . towards said teats. 

35 Advantageously, said fluid is a liquid such as 

water . 
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Preferably, in another embodiment; said fluid is 

air. 

Advantageously, said actuating device comprises 
two counter rotating brushes, 
5 Preferably, in another embodiment, the actuating 

device is both provided with spray-nozzles for spraying 
fluid towards said teats and comprises two counter rotating 
brushes . 

The location technique embodying the invention 
10 uses a laser to direct a generally horizontal sheet of 

light towards the teats of a milk animal. When viewed by 
eye, this laser light sheet will be seen as a bright, gene- 
rally horizontal line across each teat in range of the 
sheet and other such lines on other parts of the animal, 
15 such as the legs, if these are in range of the sheet. 

A video camera is directed towards the teats . The 
camera view axis may be in or above or below the sheet of 
light and is at an angle to the axis of the plane of the 
sheet. Conveniently, the camera looks upwardly through the 
20 sheet. The laser and camera are fixedly related to each 

other and move as a whole being mounted on a milking appa- 
ratus support . 

The camera receives an image generally similar to 
that seen with the eye, but in monochrome. The image is 

2 5 then processed to recover the bright line that falls across 

each teat in view. (Note: the term image is used although 
in operation no image is displayed.) One teat from the 
image is chosen, conveniently the front right teat when a 
milking robot, for example as described in GB-A-2 258 382 

3 0 approaches the udder from the right front of the animal and 

rearwardly of the animal towards the udder, and the laser 
and camera moved, under control of information derived from 
the image processing, to approach the chosen teat by move- 
ment towards the respective bright line image on the teat. 
35 This technique reliably distinguishes one teat 

from another by relative positions in the image on known 
positions. Similarly it can distinguish teats from 
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background objects such as a leg or a tail or part of a 
stall or even debris, such as straw or other vegetation 
that may be in the region of the teats, by size or shape or 
unrealistic positions. Also, the technique is relatively 
5 immune to stray light and different levels of ambient 

illumination because the line on a teat is distinguishably 
bright and of a known wavelength which may be specifically 
selected with an appropriate filter. 

In one embodiment the laser and camera are mounted 

10 on a milking apparatus support in the form of the teat -cup- 
carrier of a milking robot so that the robot can be con- 
trolled to approach a chosen teat closely enough for a 
teat -cup or other apparatus on the carrier to be reliably 
applied to the chosen teat. 

15 By repeated use of the technique on each remaining 

teat, a respective cup can be applied to each teat. Other- 
wise the position of the remaining teat or teats can be 
estimated from the established position of the chosen teat 
and stored information for the specific animal, including 

2 0 the milking history, for teat cup application without use 

of the technique once the first teat -cup has been success- 
fully applied. 

The use of the term milk animal in this 
application is inteded to be any kind of milk animal, such 
25 as cows, sheep, goats, buffaloes, and horses. 

DRAWING SUMMARY 

Embodiments of the invention will now be 
described with reference to the accompanying drawings, in 

3 0 which: 

Figure 1 shows a schematic diagram of apparatus to form a 

teat -distinguishing image , 
Figure 2 shows a block diagram of processing steps, 
Figure 3 shows a block diagram of image processing 
3 5 hardware, 

Figure 4 is a representation of the images captured by the 

camera, 
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Figure 5 is a flow chart of the control action. 
Figure 6 shows three elevations of a mounting for a laser 
and a camera, 

Figure 7 shows one example of the order in which the teats 
5 of a milk animal are selected, 

Figure 8a illustrates a first embodiment of an actuating 
device, 

Figure 8c illustrates a third embodiment of the actuating 
device, 

10 Figure 9a is a representation, similar to figure 4, of the 
images captured by the camera for explaining the 
use of the actuating device in the arrangement, 
Figure 9b is a representation, similar to figures 4 and 9a, 
of the images captured by the camera for 
15 explaining the use of the actuating device in the 

arrangement , and 
Figure 10 illustrates the movement of the teats achieved by 
the actuating device. 



2 0 DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

A particular embodiment will now be described in 
more detail . 

Figure 1 shows a small laser 10 such as a 5 milli- 
watt device from the firm Vector Technology, England, fit- 

25 ted with a lens 11 to provide a light sheet 12 of an in- 
cluded angle of some 60° to 100° with uniform illumination. 
Such a laser is a tube about 12 millimetres in diameter and 
75 millimetres long with a fairly thin connection cable, so 
it is not a severe additional load for a teat cup carrier 

30 robot arm 13, even when mounted at the outer end of the 

robot arm. The laser 10 is positioned to direct the sheet 
of light 12 a small distance above the mouth 14 of a teat- 
cup 15 when in the carrier 16 of the robot arm. 

A compact solid state camera 17 is also mounted on 

35 the teat cup carrier 16. This camera is a 1/2 inch (12 mm) 
charge coupled device camera fitted with a lens 18 to give 
a 90° angle of viewing in the vertical plane. The camera is 
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a 90° angle of viewing in the vertical plane. The camera is 
positioned at an angle to the light sheet 12 so that one 
extreme of the angle of view is on or below the mouth of 
the teat cup 14, and therefore below the light sheet 12, 
5 while the other extreme is directed to the rear of a per- 
pendicular to the plane of the light sheet. This position- 
ing of the camera view angle assists in distinguishing bet- 
ween objects at different distances from the camera. 

The video performance of the camera gives a reso- 

10 lution of approximately 400 by 400 pixels at a 25 Hz (40 

milliseconds/frame) frame rate or 400 by 200 pixels at a 50 
Hz (20 milliseconds/frame) frame rate. The 90° angle lens 
has a depth of field from 300 to 50 millimetres when back 
focussed with a small aperture (fl6) . For the camera des- 

15 cribed, this resolution is about 4 pixels/degree, approxi- 
mately 1 millimetre resolution at 200 millimetre with a 450 
millimetre field of view. A filter may be used on the came- 
ra to reduce the effect of ambient light at wavelengths 
other than that of the laser. Typically a 2% bandwidth fil- 

20 ter is suitable. 

A connection to energise the laser 10 is indicated 
at 20, signal and energisation connections for the camera 
17 at 21, and the vacuum line for the teat-cup at 22. The 
robot arm 13 can be of the type shown in GB-PS-2 226 941 or 

25 GB-PS-2 258 382 or another suitable type. The control in- 
structions for the robot arm are derived from the camera 
video information . 

An exemplary technique for deriving the control 
instructions is now described. A single frame of video from 

30 the camera is "grabbed" as a monochrome image to be pro- 
cessed and read into a memory, each pixel in the image 
being represented as an eight bit value in a memory loca- 
tion, ranging in value from 0 (black) to 255 (white) . Pre- 
ferably a single field of video may be grabbed, and pro- 

35 cessed whilst the next field is being grabbed so allowing 
real time processing. A restricted area surrounding the 
teat image, for example 50 pixels horizontally and 10 
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pixels vertically, may be used to reduce the total pixels 
scanned to 500 instead of those for a full frame (up to 512 
X 512, i.e. 262,144). The size of the restricted area is 
chosen to allow optimum tracking at a field rate of up to 
5 50 fields/second. The average and variance of the pixel 

values are calculated and used to produce a threshold value 
of brightness for the image. With this information, allow- 
ance can be made for the effects of changes in background 
(ambient) illumination and the exposure level of the came- 

10 ra. All the pixels are compared to the threshold value. If 
all the pixels below the threshold are ignored, the remain- 
ing pixels should be those in the laser stripe. These re- 
maining pixels are examined to determine whether there are 
horizontally adjacent pixels that are above the threshold. 

15 Tbose pixels that are above the threshold are grouped to- 
gether into lines which are then represented in memory as a 
linked list. (These lines are not "lines" of the image). 
Lines below a chosen (small) length are discarded, provid- 
ing a filtering of shortobjects or "glitches". Lines on 

20 successive rows of the image which overlap are grouped as 
blobs, those blobs with less than a chosen number of 
overlapping lines are discarded, and the retained blobs 
stored in a further linked list. 

A "real" geometry is now applied to the stored 

25 blobs so that the X and Y value for each blob, and its 

size, are established. Now the blobs which are unlikely to 
indicate teats, because of size, position etc., can be dis- 
carded. Various rules can be produced and applied for this 
stage, having regard to those blobs which are too narrow or 

3 0 too wide or in the wrong area under the animal to be a 
teat . 

A candidate teat is now identified from the teat 
images that have been selected by the above process. In one 
arrangement, the camera is positioned forward and on a 
35 level with a teat, analysis of the images allows correction 
of the camera image. Alternatively the camera approaches 
from the front of the udder rearwardly on the centre line 
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while the camera is moving upwards until the teat images 
are seen, the teat images are approximately symmetrical 
about the centre line, and image analysis identifies the 
selected teat. Having identified the position of a specific 
5 teat, the position of the other teats are similarly 

identified by the images. Of course they may be found by 
comparing the teat images with a reference, such as a 
"model" of average relative teat positions based on a 
sample of real animals. 

10 The software to achieve the above image processing 

is based on a mixture of proprietary and specially written 
routines. One arrangement of hardware is based on an 
ELTEC-68K system using an MC68020 processor. At 16 MHz with 
a 512 X 512 image array, an image can be processed in five 

15 seconds. This speed would only provide a rather slow teat 
application procedure while the whole image has to be pro- 
cessed, but faster processing is clearly possible with re- 
cent developments in computer speeds. Once only the re- 
stricted area has to be processed, the whole of this image 

20 area can be processed much more frequently. Figure 2 shows 
in block diagram form the sequence of image processing 
steps, while Figure 3 shows a block diagram of units of an 
image capture and processing arrangement . 

In Figure 3, a laser stripe generator is indicated 

2 5 at 31, and a camera to view an image produced by the laser 

stripe is indicated at 32. Both are on the robot arm, part 
of which is shown at 36. A power supply unit 33 energises 
electronic circuit units 34 and 35 and camera 32, if need- 
ed. Unit 35 processes the image information from camera 32 
30 and supplies image position information to unit 34 which 
provides control information at output 37 for the robot 
(not fully shown) . Power and signal cables are held to- 
gether 3 8 on the robot arm. 

In order to produce a cost-effective technique for 

3 5 guiding a milking apparatus support towards a teat of a 

milk animal, the processing of the image received by the 
camera must be as efficient as possible. With unlimited 
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computational power and speed, it would be possible to 
process a complete image quickly enough to produce guidance 
information for a robot operating the milking apparatus 
support to follow any teat movement in real time. 
5 However such an arrangement is an extravagant use 

of resources and would greatly increase the cost and com- 
plexity of the equipment. 

By way of an example of the invention, one tech- 
nique for guidance employing the equipment such as that 

10 described above, which significantly reduces processing 

needs while still providing adequate real time guidance in- 
formation for the robot, will now be described with refe- 
rence to Figure 4 . 

Consider the image 4 3 formed by the camera as the 

15 sheet of light from the support is moved by movement of the 
robot towards the expected general position of the udder 
and teats 44,45,56,58. In this example, the sheet is moved 
upwardly and rearwardly beneath the animal . Because of the 
geometry of the camera and light sheet, an object il- 

2 0 luminated by the moving sheet will move more quickly up the 
image the nearer the object is to the camera. The image 
includes a notional fixed point 50 which represents a posi- 
tion just above the supported teat cup. The point 50 is 
fixed in position in the image 41, generally at its centre, 

2 5 because it is part of the "real" geometry of the laser, 

camera and teat cup support. The point 50 is conveniently 
the place at which a teat will enter a teat cup when the 
supported teat cup approaches from below, assisted if ap- 
propriate by the conventional vacuum. The extremes of the 
30 image 43 are not illuminated by the laser. Image portion 41 
is the one defined by the "real" geometry as the expected 
place for a teat. The area of image 43 above image portion 
41 is too near to include a teat, and that area below 41 is 
too far away to do so. The image portion 41 is geometrical- 

3 5 ly linked to that of the support on the robot, and the re- 

gion shown in the image changes with robot movement. 
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The image portion 41 is examined by scanning the 
image pixels in columns 40 between the top and bottom (or 
vice versa) of the image, across the image. The scanning is 
performed by moving the robot arm upwards and downwards . 
5 The laser and camera are being stationary arranged on said 
robot arm. The brightest pixel in each column is noted 
47,48,49,57,59. When a group of such pixels side-by-side in 
adjacent columns and maintained in successive scans is 
found, this is considered as a possibility for the image of 

10 the light on the teat nearest the camera. {A single bright 
pixel is postulated for each column. In practice, a "stick" 
of a small number of pixels vertically adjacent in a column 
may be found whose brightness increases along the stick to 
a peak or plateau, which may be saturation, and then 

15 reduces. The peak or the middle of the plateau is taken as 
the significant pixel position.) The groups 48,49,57,59 
would be identified as possible teat candidates, while the 
group 4 7 would be rejected as too large for a teat, arising 
from a leg 46. The group 49 would then be selected as a 

20 target teat, as being the nearest teat. Pixels representing 
reflections on hoses or parts of the stall could be 
rejected in a similar manner or by software techniques. 

Once a suitable group 4 9 of bright side-by-side 
pixels has been identified as representing a teat of inte- 

25 rest, a target teat 44, a subimage 42, some 30 by 100 

pixels for example, of the image portion and which encloses 
the identified pixel group is defined, and only this sub- 
image is then scanned and processed to provide guidance in- 
formation to the robot to move the supported teat cup, or 

30 other apparatus, towards the deduced position of the pixel 
group. This subimage 42 is within image portion 41 but not 
fixed in position within image portion 41. Consider the 
support, light source and camera moving through the region 
under the animal. Now consider the image portion 41. This 

3 5 will have at its centre, or some other known chosen point, 
the fixed point 50. Image portion 41 will also include sub- 
image 42 which includes the image of the teat of interest 
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but probably does not include point 50. The guidance infor- 
mation is derived from the relative positions of point 50 
and subimage 42 and more specifically the position of the 
centre of the teat image 4 9 therein. Thus, in successive 
5 images examined by the equipment, the effect of the guid- 
ance information will be to move the centre of the teat 
image towards the point 50. By limiting the scanning and 
processing to only the subimage, the response of the robot 
can be improved. Referring again to Figure 4, the situation 

10 shown is the one just described, with the subimage 42, and 
thus the centre of teat image 49, at some distance from 
point 50. Notionally, image 41 has three distinct areas, 
51, 52 and 53. Area 53 is the area immediately around point 
50 and is indicated by boundary 55. Area 52 is around area 

15 53 and is indicated by boundary 54. Area 51 is around area 
52 and, as shown, is bounded by the edges of image 41. Of 
course more or fewer such areas can be used. The centre of 
image 4 9 is thus in the area 51, which is most remote from 
point 50. Accordingly, the guidance information for the 

20 robot derived from the subimage is provided at a high con- 
trol gain so that the position of the robot is quickly cor- 
rected, towards the teat image centre, because the "error" 
is large. As the robot approaches the teat of interest by 
this continued correction, the teat image centre will move 

25 from area 51 into area 52, crossing boundary 54. Guidance 

information is now supplied at a lower control gain so that 
the robot approaches less quickly. When the teat image 
centre by continued correction enters area 53, the control 
gain and robot approach are further reduced. In this way, 

30 the tracking is adjusted to the approach of the robot to 

the teat of interest, so that the possibility of the robot 
"losing" the teat by "hunting" or overshoot is reduced. The 
fixed point 50 is actually associated with the robot arm, 
so when the robot arm is moving, the point 50 is also 

3 5 moving in the image portion 41. When e.g. the robot arm 
has moved towards the teat 44 the fixed point 50 will now 
be situated in the sub-image 42. Conveniently, area 52 is 



WEST 



wo 97/15900 



PCT/SE96/0138I 



16 



sized by boundary 54, so point 50 is some 50 millimetres 
from the identified teat position. Area 53 is sized so that 
point 50 is some 20 millimetres from the identified teat 
position. The image 43 captured by the camera is 512 by 512 
5 pixels or 256 by 512 pixels. Typically, the image portion 
41 is 400 by 400 pixels respectively, processed at 25 
frames per second, or 200 pixels wide by 400 pixels high, 
processed at 50 frames per second, and the subimage 42 is a 
total of 3000 pixels of suitable rectangular shape, such as 

10 100 pixels wide by 30 pixels high, and processed at 40 or 
5 0 frames per second. By using machine code for the image 
processing, the whole of the image portion 41 can be 
processed in 200 ms . This is fast enough to "capture" a 
possible teat image for the subimage. Also typically, a 

15 minimum of eight pixels side -by- side are considered 

appropriate for a group to be treated as a "possible" teat 
image. When the subimage is selected from the image, the 
repetition rate is increased and the subimage can be 
processed in seven milliseconds, which gives a repetition 

20 rate fast enough to track a teat when moved by normal 
animal movements. If required, a portion of image 41, 
intermediate portion 41, and subimage 42 can be processed 
as an intermediate step. 

It is likely that the animal will move during the 

25 approach of the robot. It is estimated that the typical 
speed of movement of a teat will be some 100 millimetres 
per second unless a very violent movement occurs. If the 
movement is so violent that the teat image moves out of the 
image portion in use at the time, the approach procedure 

30 can be terminated either immediately or after a further 
period unless a suitable image is recaptured. 

Similar techniques are usable for other milking 
apparatus such as teat cleaning and teat inspection de- 
vices . 

35 Figure 5 is a flow chart of the method of guiding 

a milking apparatus support towards a teat of a milk animal 
according to th present invention. The method begins at 
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block 70. At block 12, the method continues with moving 
said support to a fixed start position, without any 
reference to any animal. This fixed start position can be 
any suitable position, e.g. in a stable. This fixed 
5 position does not have any reference to, nor does it depend 
on the position of any animal. The method continues at 
block 74 with illuminating, by a sheet of light from the 
support, a region expected to contain at least one udder. 
The next step, at block 76, consists of capturing from said 

10 support images by viewing the region with a video camera. 
The method continues at block 78 with analysing said 
captured images in order to identify possible teat 
candidates (see fig. 4; 44, 45,56,58). The next step, at 
block 80, consists of determining if any teat candidates 

15 have been found. If the answer is negative, i.e. no teat 
candidates have been found, the method continues at block 
82 with moving the support and repeating the steps, 
starting with block 74. If, on the other hand, the answer 
is affirmative the method continues at block 84. This step 

20 consists of selecting one of said teat candidates as a 

target teat (see fig. 4; 44). The method continues at block 
86 with determining the position of said target teat. The 
next step, at block 88, consists of homing in said support 
and any supported milking apparatus to said target teat . 

2 5 The next step, at block 90, consists of determining if each 

of the four teats of a milk animal have been selected as a 
target teat and said support and any supported milking 
apparatus have been homed in to each of said four teats. 
If the answer is negative, the steps starting with block 74 

3 0 are repeated. If, on the other hand, the answer is 

affirmative the method continues at block 92, where the 
method is completed. 

Figure 6 shows in views 6a rear elevation, 6b side 
elevation, 6c front elevation, a constructional arrangement 
3 5 of a mounting 60, by which a laser and camera can be mount- 
ed on a robot. A suitable camera is a Pacific Corporation 
type VPC 465, which is a 40 millimetre square unit in an 
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laser can be as described above or a semiconductor red 
laser which is smaller. Mounting 60 is a block in the form 
of a rectangular prism with part cut away to give an in- 
5 clined face 61. The block is 60 millimetres square in plan 
and about 80 millimetres high. Face 61 is inclined at about 
67° to the base. A bore. 62 is parallel to base and emerges 
near the top of the face 61. Bore 62 is sized to receive 
the laser. An aperture 63 is provided to receive the camera 
10 clear of the laser. A bore 64 is provided for the camera 
connection. Conveniently, the block is of aluminium or a 
sufficiently rigid other light material, even a plastics 
material. Suitable tapped holes are provided for fasten- 
ings . 

15 Figure 7 shows one example of the order in which 

the teats of a milk animal 100 are selected. The milk 
animal 100 is only shown partially and schematically from 
above and only the rear part of the milk animal 100 is 
shown. The four teats 102, 104, 106, 108 of the milk 

20 animal 100 are also indicated in the figure. Schematically 
indicated in the figure are also four teat cups 110, 112, 
114, 116 arranged in a row alongside of said milk animal 
100. Assume that the robot arm with said support (see 
figure 1) is situated alongside of said milk animal 100 but 

25 at the opposite side in relation to the four teat cups 110, 
112, 114, 116. The robot arm starts e.g. by fetching teat 
cup 110 and aplying it on the teat 102, Then the robot arm 
fetches the teat cup 112 and applies it on the teat 104 and 
then continues with applying the teat cup 114 on the teat 

30 106 and finally applies the teat cup 116 on the teat 108. 

It is obvious that the robot arm and the four teat cups can 
be situated in many other ways. The most important thing 
is that the order in which the teats of the milk animal are 
selected is performed in such a way that already applied 

35 teat cups do not interfere with the application of the rest 
of the teat cups. 
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Figure 8 a, illustrates a first embodiment of an 
actuating device 120 for actuating a region containing said 
possible teat candidates, so as to bring at least the teats 
to movement. In the illustrated embodiment the actuating 
5 device 120 is arranged on a robot arm separate from the 
robot arm 13 (see fig. 1) carrying the teat cup. This 
actuating device 120 comprises two counter rotating brushes 
122; 124, between which the teat is intended to be 
inserted. 

10 Figure 8b, illustrates a second embodiment of the 

actuating device 130 for actuating a region containing said 
possible teat candidates, so as to bring at least the teats 
to movement. In this embodiment the actuating device 130 is 
arranged on the carrier 16 of the robot arm 13 carrying the 

15 teat cup 14. This actuating device 130 is provided with a 
plurality of spray-nozzles 132 (only one spray-nozzle is 
illustrated) for spraying a fluid towards said teats. The 
fluid can either be a liquid, e.g. water, or a gas medium, 
e.g. air. 

20 Figure 8c, illustrates a third embodiment of the 

actuating device 150 for actuating a region containing said 
possible teat candidates, so as to bring at least the teats 
to movement. In this embodiment the actuating device 150 is 
arranged on a post 152 of a stall, containing a cow 154. 

25 This actuating device 150 is provided with a plurality of 
spray- -nozzles 156 (only one spray-nozzle is illustrated) 
for spraying a fluid towards said teats. The fluid can 
either be a liquid, e.g water, or a gas medium, e.g. air. 

The actuating device can also be a combination of 

30 the above disclosed embodiments, i.e. the actuating device 
can be provided with a plurality of spray-nozzles and two 
counter rotating brushes for spraying liquid and/or gas 
towards the teats and mechanically moving and cleaning the 
teats . 

35 In the embodiments disclosed in figures 8 a-c, the 

actuating device 120; 130; 150 is arranged on a robot arm 
separate from the robot arm 13 (see fig. i) carrying the 
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teat cup; on the carrier 16 of the robot arm 13 carrying 
the teat cup 14; or on a post 152 of a stall. Obviously 
the actuating device 12 0; 13 0; 150 can be arranged on the 
floor in the stall. 
5 The main advantage of using an actuating device in 

accordance with the description above is that it is easier 
to identify the teats while they are moving, or after they 
have performed a movement. It is easier to distinguish 
them from other sources of error such as the animal's legs 

10 and equipment in the stall. Another advantage is that 

straws can be swept away resulting in the elimination of a 
source of error. 

Figure 9a is a representation, similar to figure 
4, of the images captured by the camera for explaining the 

15 use of the actuating device in the arrangement according to 
the invention. As is apparent figure 9a is similar to 
figure 4 and like reference numerals designate like 
structural elements. Accordingly, the descriptive part 
corresponding to figure 4 will not be repeated again. 

2 0 Consider the image 43 formed by the camera, as the sheet of 
light from the support is moved by movement of the robot 
towards the expected general position of the udder and 
teats 44, 45, 56, 58. The image portion 41 is examined by 
scanning the image pixels 40 between the top and bottom of 

25 the image, across the image. The brightest pixel in each 
column is noted 47, 49, 57, 59, 142, 146. When a group of 
such pixels side-by-side in adjacent columns and maintained 
in successive scans is found, this is considered as a 
possibility for the image of the light on the teat nearest 

30 the camera. In the worst case, the groups 47, 49, 57, 59, 
142, 146 would be identified as possible teat candidates. 
In this example the groups 49, 57, 59 arises from the teats 
45, 56, 58, the group 47 arises from a leg 46, the group 
142 arises from a post 140 in the stall and the group 146 

35 arises from a straw 144. This means that the system can 

miss the teat 44, because the straw 144 is identified as a 
teat candidate. In this situatuion a control means sends a 



WEST 



wo 97/15900 



PCT/SE96/01381 



21 



signal to the actuating device 120/130, whereby the actuat- 
ing device 120; 130 moves towards the region containing the 
seven teat candidates. Assume, e.g., that the actuating 
device 120 comprises two counter rotating brushes 122, 124 
5 (see figure 8a) . When the actuating device comes into 

contact with the post 140, the post 140 will not move, and 
consequently the group 142 will not move. The group 146 
will disappear if the straw 144 comes off the teat 4 5 when 
the actuating device 120 comes into contact with the teat 

10 45 and the straw 144. When the straw 144 comes off the teat 
45 the system will capture the group 48' as a possible teat 
candidate. When the actuating device 120 actuates the teats 
44, 45, 56, 58 they will start moving. They are identified 
as teats by establishing said movement. 

15 In another embodiment of the invention the teats 

are identified by establishing characteristic movement 
thereof. The teats are moving in a characteristic way, i.e. 
with a frequency within an interval of identification for 
teats in general. This means that the groups 48, 49, 57, 

20 59 also will move in this characteristic way. When the 
actuating device 120 actuates the leg 46, the leg 46 or 
rather the group 47 will not move, but if it moves, the 
group 47 arising from the leg 46 will be moving with 
another "frequency" or another movement pattern than the 

25 groups 44, 45, 56, 58. 

Figure 9b is a representation of the images 
captured by the camera for explaning the use of the 
actuating device in the arrangement according to the 
invention. As is apparent figure 9b is similar to figures 

30 4 and 9a and like reference numerals designate like 

structural elements. In this figure a straw 144 is situated 
on the teat 45 so the group 146 will be rather broad. In 
the worst case the group 146 will be inter-preted as too 
large for a teat. Again the actuating device will sweep 

3 5 away the straw 144 and the system will identify the teat 4 5 
as described above. 
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Figure 10 illustrates the movement of the teats 
achieved by the actuating device. For the reason of 
simplicity only one teat 45 is shown. The situation shown 
is just after the actuating device 120; 130 has actuated the 
teat 45 and the straw 144. The straw 144 has disappeared 
from the teat 45 and the teat 4 5 will be moving forth and 
back in accordance with the arrows A, with a characteristic 
frequency. The control means can now distinguish the teats 
among all the teat candidates. 

The above techniques provide an effective and ef- 
ficient arrangement for the application of milking appara- 
tus to a milk animal. 
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CLAIMS 

1. A method of guiding a milking apparatus support 
(16) towards at least one teat of a milk animal, character- 

5 ized in that the method comprises the following steps: 
to move said support (16) to a fixed start position 
within a stall, without any reference to any animal, 
to illuminate (10,11) with a sheet of light (12) from 
the support (16) a region expected to contain at least 

10 one udder, 

to capture from said support (16) images (41) by 
viewing the region with a video camera (17) , 
to analyze said captured images (41) to identify pos- 
sible teat candidates (44,45,56,68), 

15 - to select one of said teat candidates (44,45,56,58) as 
a target teat (44), 

to determine the position of said target teat (44), and 
to home in said support (16) and any supported milking 
apparatus (15) to said target teat (44). 

20 

2. A method according to claim 1, characterized in 
that the method, subsequently to selecting a target teat 
(44), comprises the step: 

to analyze only a subimage (4 2) of said captured image 
25 (41), which subimage, (42) includes said target teat 

(44) . 

3. A method according to claim 2, characterized in 
that the method also comprises the following steps: 

30 - to provide in said captured image (41) the position 

(50) of a teat cup entry point for a target teat (44), 
to quantify the separation of the teat cup entry point 
from the position of said target teat (44), and 
to provide homing in information for said support (16) 

35 on the basis of the amount of said separation. 
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4. A method according to claim 3, characterized in 
that the method also comprises the step: 

to quantify said separation in steps defined by areas 
(51,52,53) of said captured image. 

5 

5. A method according to claim 4, characterized in 
that the method also comprises the step: 

to guide the support (16) to move the subimage (4 2) of 
the captured image (41) within the captured image (4i) 
10 to where the entry point for a supported teat cup (15) 

is at the image position (50) of the teat cup entry 
point for the selected teat cup (15) . 

6. A method according to claim 5, characterized in 
15 that the method also comprises the step: 

to identify possible teat candidates (44,45,56,58) as 
respective pixels which are adjacent and having a form 
and position appropriate to a teat (44) illuminated by 
the light (12) . 

20 

7 . A method according to claim 6 , characterized in 
that the method also comprises the step: 

to identify possible teat candidates (44,45,56,58) by 
scanning the image (41) in a succession of vertical 
25 scans (40) to identify the brightest pixel in each 

scan. 

8. A method according to claim 7, characterized in 
that the method also comprises the step: 

30 - to identify the brightest pixel from among a vertical 
line of pixels of differing brightness. 

9. A method according to claim 7, characterized in 
that the method also comprises the step: 

35 - to identify a possible teat candidate (44,45,56,58) 

from said brightest pixels in adjacent scans which form 
a substantially horizontal line of length in a pre- 
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selected range of pixels. 

10. A method according to any one of the preceding 
claims, characterized in that the method also comprises the 

5 steps : 

to actuate a region containing said possible teat 
candidates with the aid of an actuating device 
(120;130;150) , SO as to bring at least one teat to 
movement , and 

10 - to identify said at least one teat by establishing said 
movement . 

11. A method according to claim 10, characterized in 

that said actuating step is performed by having said 
15 actuating device (120) making physical contact with said 
teat candidates. 

12. A method according to claim 10, characterized in 
that said actuating step is performed by having a fluid 

20 sprayed towards said teat candidates. 

13. A method according to claim 10, characterized in 
that said actuating step is performed by having both a 
fluid sprayed towards said teat candidates and said 

25 actuating device (120) making physical contact with said 
teat candidates. 

14. A method according to any one of claims 10-13, 
characterized in that said identifying step is performed by 

30 identifying said at least one teat by establishing charac- 
teristic movement thereof. 

15. A method according to any one of the preceding 
claims, characterized in that the method also comprises the 

35 step: 
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to repeat the above mentioned steps, starting 
with the illuminating step, until each of all the 
teats of a milk animal have been selected as a 
target teat (44) , and said support (16) and any 
5 supported milking apparatus (15) have been homed 

in to each of all the teats. 



16. A milking apparatus support (16) guide arrange- 
ment, characterized in that the support (16) carries a 

10 source (10,11) of a sheet of light (12), positioned to pass 
no lower than the mouth (14) of a teat cup (15) arranged on 
said support (16), and a video camera (17) arranged to view 
through the sheet and over the mounth (14) of the teat cup 
(15), wherein said source (10,11) and video camera (17) are 

15 together arranged to cooperate (12,19) forwardly of the 

support (16), the video camera (17) arranged to capture an 
image (41) formed by said light (12) forwardly of the 
support (16) and provide an image signal, the arrangement 
further including image signal processing means (35) to 

20 analyze said captured image signal and identify possible 
teat candidates (44,45,56,58), to select one of said teat 
candidates (44,45,56,58) as a target teat (44) and to de- 
termine the position of said target teat (44) to home in 
said support (16) and any supported milking apparatus (15) 

25 to said target teat (44) . 

17. An arrangement according to claim 16, character- 
ized in that the image signal processing means (35) 
includes means to select from the captured image signal 

30 (41) a subimage (42) including said target teat (44) and 

analyze only said subimage (42), thereby reducing the time 
between successive processings of said subimage (42) . 

18. An arrangement according to claim 17, character- 
35 ized in that the image signal processing means (35) pro- 
vides in said captured image (41) the position (50) of a 
teat cup entry point for a target teat (44) . 
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19. An arrangement according to claim 18, character- 
rized in that the image signal processing means (35) in- 
cludes means to quantify the spatial separation of the teat 
cup entry point from the position of said target teat (44) 
and provide homing in information for said support (16) on 
the basis of the amount of said separation. 

20. An arrangement according to claim 19. character- 
ized in that the homing in information causes more rapid 
movement of the support (16) the greater said spatial 

separation . 

21. An arrangement according to any one of the 
preceding claims, characterized in that the arrangement 
also comprises an actuating device (120 ; 130 ; 150) for ac- 
tuating a region containing said possible teat candidates, 
so as to make at least one teat moving, and a control means 
to identify said at least one teat by establishing the 
movement . 

22. An arrangement according to claim 21, character- 
ized in that said actuating device (120;130) is arranged on 
said support (16) . 

25 23 An arrangement according to claim 21, character- 

ized in that said actuating device (120; 130) is arranged on 
a separate moveable support . 



15 



20 



30 



24. An arrangement according to claim 21, character- 

ized in that said actuating device (120;130) is arranged on 
the floor in a stall. 



25, An arrangement according to claim 21, character- 

ized in that said actuating device (120; 150) is arranged on 
35 a post of a stall. 
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26. An arrangement according to any of claims 21-15, 
characterized in that said actuating device (130; 150) is 
provided with spray-nozzles (132; 156) for spraying a fluid 
towards said teats. 

5 

27. An arrangement according to claim 26, character- 
ized in that said fluid is a liquid, such as water. 

28. An arrangement according to claim 26, character- 
10 ized in that said fluid is air. 

29. An arrangement according to any of claims 21-25, 
characterized in that said actuating device (120) comprises 
two counter rotating brushes (122; 124) 

15 

30. An arrangement according to any of claims 21-25, 
characterized in that said actuating device both is 
provided with spray-nozzles for spraying fluid towards said 
teats and two counter rotating brushes. 

20 
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